Clipping view

ABSTRACT

System and process for handling clips with annotations. A user annotates a document that is displayed in an access session. Information is gathered regarding the annotation, which is associated with active content included in the document. The annotation is stored with a static image of the active content. A link to the content is stored. An image is rendered that has a clip including the stored annotation and the image of the content, and is displayed.

RELATED APPLICATIONS

This application is a continuation of and claims benefit of priority toU.S. patent application Ser. No. 10/777,046, filed on Feb. 13, 2004,which application is herein incorporated by reference.

BACKGROUND

Aspects of the present invention relate to image processing andinformation manipulation. More specifically, aspects of the presentinvention relate to obtaining and using context information related toselected content

DESCRIPTION OF RELATED ART

People mark-up documents. For electronic documents, people commonlyprint them out and annotate them with highlighters, pens, tape flags,and the like. While a document has significance by itself, the readerhaving made annotations has effectively indicated which sections are themost relevant to him. These annotations are generally graphical innature and do not always transition well to a textual equivalent.

In the computing world, however, attempting to capture annotations andrelated markups in an electronic fashion can be cumbersome. Typicalcomputer systems do not provide an easy interface for capturing andconveying graphically intensive content. Rather, they are optimized forcapturing and rendering text. For instance, typical computer systems,especially computer systems using graphical user interface (GUI)systems, such as Microsoft WINDOWS, are optimized for accepting userinput from one or more discrete input devices such as a keyboard forentering text, and a pointing device such as a mouse with one or morebuttons for driving the user interface.

Some computing systems have expanded the input and interaction systemsavailable to a user by allowing the use of a stylus to input informationinto the systems. The stylus may take the place of both the keyboard(for data entry) as well as the mouse (for control). Some computingsystems receive handwritten electronic information or electronic ink andimmediately attempt to convert the electronic ink into text. Othersystems permit the electronic ink to remain in the handwritten form.

Despite the existence of a stylus, providing a user's annotations andother markups back to a user in an efficient manner is difficult. Whileone may view thumbnails of a document with annotations, thecorresponding annotations and related content is shrunk to fit withineach thumbnail. For annotation and markup heavy documents, thethumbnails may be useless. Accordingly, a better solution for providingcontent to a user is needed.

SUMMARY

Aspects of the present invention address one or more of the issuesmentioned above, thereby providing a better presentation of annotationsand their associated context. Various aspects of the invention mayinclude at least one of combining and filtering with a resulting displayof the annotations.

These and other aspects are addressed in relation to the Figures andrelated description.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a general-purpose computer supporting one or more aspectsof the present invention.

FIG. 2 shows a display for a stylus-based input system according toaspects of the present invention.

FIG. 3 shows a document with content and various annotations inaccordance with aspects of the present invention.

FIG. 4 shows information associated with each annotation being gatheredin accordance with aspects of the present invention.

FIG. 5 shows grouping of annotations in accordance with aspects of thepresent invention.

FIG. 6 shows annotations on electronic ink being grouped in accordancewith aspects of the present invention.

FIGS. 7 and 8 show processes for combining and displaying annotations inaccordance with aspects of the present invention.

FIGS. 9 and 10 show orderings of clips of annotations in accordance withaspects of the present invention.

FIG. 11 shows various levels of links in accordance with aspects of thepresent invention.

FIG. 12 shows an illustration of active content being annotated inaccordance with aspects of the present invention.

FIG. 13 shows a user interface in accordance with aspects of the presentinvention.

FIG. 14 shows information stored with relation to an annotation inaccordance with aspects of the present invention.

DETAILED DESCRIPTION

Aspects of the present invention relate to obtaining and displayingannotations in brief format.

This document is divided into sections to assist the reader. Thesesections include: characteristics of ink, terms, general-purposecomputing environment, annotation and clip handling, processes forhandling annotations and clips, clips and links, user interfaces, anddata structures.

Characteristics of Ink

As known to users who use ink pens, physical ink (the kind laid down onpaper using a pen with an ink reservoir) may convey more informationthan a series of coordinates connected by line segments. For example,physical ink can reflect pen pressure (by the thickness of the ink), penangle (by the shape of the line or curve segments and the behavior ofthe ink around discreet points), and the ˜_(peed) of the nib of the pen(by the straightness, line width, and line width changes over the courseof a line or curve). Because of these additional properties, emotion,personality, emphasis and so forth can be more instantaneously conveyedthan with uniform line width between points.

Electronic ink (or ink) relates to the capture and display of electronicinformation captured when a user uses a stylus-based input device.Electronic ink refers to a sequence of strokes, where each stroke iscomprised of a sequence of points. The points may be represented using avariety of known techniques including Cartesian coordinates (X, V),polar coordinates (r, Θ), and other techniques as known in the art.Electronic ink may include representations of properties of real inkincluding pressure, angle, speed, color, stylus size, and ink opacity.Electronic ink may further include other properties including the orderof how ink was deposited on a page (a raster pattern of left to rightthen down for most western languages), a timestamp (indicating when theink was deposited), indication of the author of the ink, and theoriginating device (at least one of an identification of a machine uponwhich the ink was drawn or an identification of the pen used to depositthe ink) among other information.

Terms

Ink—A sequence or set of strokes with properties. A sequence of strokesmay include strokes in an ordered form. The sequence may be ordered bythe time captured or by where the strokes appear on a page or incollaborative situations by the author of the ink. Other orders arepossible. A set of strokes may include sequences of strokes or unorderedstrokes or any combination thereof. Further, some properties may beunique to each stroke or point in the stroke (for example, pressure,speed, angle, and the like). These properties may be stored at thestroke or point level, and not at the ink level

Ink object—A data structure storing ink with or without properties.

Stroke—A sequence or set of captured points. For example, when rendered,the sequence of points may be connected with lines. Alternatively, thestroke may be represented as a point and a vector in the direction ofthe next point. In short, a stroke is intended to encompass anyrepresentation of points or segments relating to ink, irrespective ofthe underlying representation of points and/or what connects the points.

Point—Information defining a location in space. For example, the pointsmay be defined relative to a capturing space (for example, points on adigitizer), a virtual ink space (the coordinates in a space into whichcaptured ink is placed), and/or display space (the points or pixels of adisplay device).

Document—Any electronic file that has a viewable representation andcontent. A document may include a web page, a word processing document,a note page or pad, a spreadsheet, a visual presentation, a databaserecord, image files, and combinations thereof.

General-Purpose Computing Environment

FIG. 1 illustrates a schematic diagram of an illustrative conventionalgeneral-purpose digital computing environment that can be used toimplement various aspects of the present invention. In FIG. 1, acomputer 100 includes a processing unit 110, a system memory 120, and asystem bus 130 that couples various system components including thesystem memory to the processing unit 110. The system bus 130 may be anyof several types of bus structures including a memory bus or memorycontroller, a peripheral bus, and a local bus using any of a variety ofbus architectures. The system memory 120 includes read only memory (ROM)140 and random access memory (RAM) 150.

A basic input/output system 160 (BIOS), containing the basic routinesthat help to transfer information between elements within the computer100, such as during start-up, is stored in the ROM 140. The computer 100also includes a hard disk drive 170 for reading from and writing to ahard disk (not shown), a magnetic disk drive 180 for reading from orwriting to a removable magnetic disk 190, and an optical disk drive 191for reading from or writing to a removable optical disk 192 such as a CDROM or other optical media. The hard disk drive 170, magnetic disk drive180, and optical disk drive 191 are connected to the system bus 130 by ahard disk drive interface 172, a magnetic disk drive interface 193, andan optical disk drive interface 194, respectively. The drives and theirassociated computer-readable media provide nonvolatile storage ofcomputer readable instructions, data structures, program modules andother data for the personal computer 100. It will be appreciated bythose skilled in the art that other types of computer readable mediathat can store data that is accessible by a computer, such as magneticcassettes, flash memory cards, digital video disks, Bernoullicartridges, random access memories (RAMs), read only memories (ROMs),and the like, may also be used in the example operating environment.

A number of program modules can be stored on the hard disk drive 170,magnetic disk 190, optical disk 192, ROM 140 or RAM 150, including anoperating system 195, one or more application programs 196, otherprogram modules 197, and program data 198. A user can enter commands andinformation into the computer 100 through input devices such as akeyboard 101 and pointing device 102. Other input devices (not shown)may include a microphone, joystick, game pad, satellite dish, scanner orthe like. These and other input devices are often connected to theprocessing unit 110 through a serial port interface 106 that is coupledto the system bus, but may be connected by other interfaces, such as aparallel port, game port or a universal serial bus (USB). Further still,these devices may be coupled directly to the system bus 130 via anappropriate interface (not shown). A monitor 107 or other type ofdisplay device is also connected to the system bus 130 via an interface,such as a video adapter 108. In addition to the monitor, personalcomputers typically include other peripheral output devices (not shown),such as speakers and printers. In one embodiment, a pen digitizer 165and accompanying pen or stylus 166 are provided in order to digitallycapture freehand input. Although a direct connection between the pendigitizer 165 and the serial port interface 106 is shown, in practice,the pen digitizer 165 may be coupled to the processing unit 110directly, parallel port or other interface and the system bus 130 by anytechnique including wirelessly. Also, the pen 166 may have a cameraassociated with it and a transceiver for wirelessly transmitting imageinformation captured by the camera to an interface interacting with bus130. Further, the pen may have other sensing systems in addition to orin place of the camera for determining strokes of electronic inkincluding accelerometers, magnetometers, and gyroscopes.

Furthermore, although the digitizer 165 is shown apart from the monitor107, the usable input area of the digitizer 165 may be co-extensive withthe display area of the monitor 107. Further still, the digitizer 165may be integrated in the monitor 107, or may exist as a separate deviceoverlaying or otherwise appended to the monitor 107.

The computer 100 can operate in a networked environment using logicalconnections to one or more remote computers, such as a remote computer109. The remote computer 109 can be a server, a router, a network PC, apeer device or other common network node, and typically includes many orall of the elements described above relative to the computer 100,although only a memory storage device 111 has been illustrated inFIG. 1. The logical connections depicted in FIG. 1 include a local areanetwork (LAN) 112 and a wide area network (WAN) 113. Such networkingenvironments are commonplace in offices, enterprise-wide computernetworks, intranets and the Internet.

When used in a LAN networking environment, the computer 100 is connectedto the local network 112 through a network interface or adapter 114.When used in a WAN networking environment, the personal computer 100typically includes a modem 115 or other means for establishing acommunications over the wide area network 113, such as the Internet. Themodem 115, which may be internal or external, is connected to the systembus 130 via the serial port interface 106. In a networked environment,program modules depicted relative to the personal computer 100, orportions thereof, may be stored in the remote memory storage device.Further, the system may include wired and/or wireless capabilities. Forexample, network interface 114 may include Bluetooth, SWLan, and/or IEEE802.11 class of combination abilities. It is appreciated that otherwireless communication protocols may be used in conjunction with theseprotocols or in place of these protocols.

It will be appreciated that the network connections shown areillustrative and other techniques for establishing a communications linkbetween the computers can be used. The existence of any of variouswell-known protocols such as TCP/IP, Ethernet, FTP, HTTP and the like ispresumed, and the system can be operated in a client-serverconfiguration to permit a user to retrieve web pages from a web-basedserver. Any of various conventional web browsers can be used to displayand manipulate data on web pages.

FIG. 2 illustrates an illustrative tablet PC 201 that can be used inaccordance with various aspects of the present invention. Any or all ofthe features, subsystems, and functions in the system of FIG. 1 can beincluded in the computer of FIG. 2. Tablet PC 201 includes a largedisplay surface 202, e.g., a digitizing flat panel display, preferably,a liquid crystal display (LCD) screen, on which a plurality of windows203 is displayed. Using stylus 204, a user can select, highlight, and/orwrite on the digitizing display surface 202. Examples of suitabledigitizing display surfaces 202 include electromagnetic pen digitizers,such as Mutoh or Wacom pen digitizers. Other types of pen digitizers,e.g., optical digitizers, may also be used. Tablet PC 201 interpretsgestures made using stylus 204 in order to manipulate data, enter text,create drawings, and/or execute conventional computer application taskssuch as spreadsheets, word processing programs, and the like.

The stylus 204 may be equipped with one or more buttons or otherfeatures to augment its selection capabilities. In one embodiment, thestylus 204 could be implemented as a “pencil” or “pen”, in which one endconstitutes a writing portion and the other end constitutes an “eraser”end, and which, when moved across the display, indicates portions of thedisplay are to be erased. Other types of input devices, such as a mouse,trackball, or the like could be used. Additionally, a user's own fingercould be the stylus 204 and used for selecting or indicating portions ofthe displayed image on a touch-sensitive or proximity-sensitive display.Consequently, the term “user input device”, as used herein, is intendedto have a broad definition and encompasses many variations on well-knowninput devices such as stylus 204. Region 205 shows a feedback region orcontact region permitting the user to determine where the stylus 204 hascontacted the display surface

In various embodiments, the system provides an ink platform as a set ofCOM (component object model) services that an application can use tocapture, manipulate, and store ink. One service enables an applicationto read and write ink using the disclosed representations of ink. Theink platform may also include a mark-up language including a languagelike the extensible markup language (XML). Further, the system may useDCOM as another implementation. Yet further implementations may be usedincluding the Win32 programming model and the .Net programming modelfrom Microsoft Corporation.

Annotation and Clip Handling

FIG. 3 shows a document with content and various annotations inaccordance with aspects of the present invention. Document 301 includeselectronic ink represented by ink strokes 302-304, image datarepresented by picture 308, and text information represented by text310.

Electronic ink 302 has been highlighted by highlights 305. Highlights305 may be highlighting ink, electronic ink in a different color thatmay be used to emphasize other content. Electronic ink 303 has beenunderlined as represented by underlining 306. Electronic ink 304 hasbeen annotated with vertical bars 307 in the right margin. Image 308 hasbeen encircled by loop 309. Text 310 has been annotated by some wordshaving been circled with loop 311 and an ink note 312 added. Otherannotations and underlying document contents are possible. Those shownin FIG. 3 are for illustrative purposes only.

FIG. 4 shows information associated with each annotation being gatheredin accordance with aspects of the present invention. Each annotation inFIG. 3 relates to content as described above. FIG. 4 shows thedesignation of context information relevant to each annotation. Forinstance, highlight 305 and a portion of ink 302 is designated by region401. Region 401 may be a rectangle or any other shape. In general,region 401 is referred to as a bounding box in that it reflects thebounds of the combination of the annotation and underlying content.

A bounding box mayor may not encompass all related content. Forinstance, for highlight 305, some of ink 302 may be captured in boundingbox 401. Alternatively, all of ink 302 may be captured in bounding box401. Bounding box 401 may be resized as needed to include as much or aslittle of underlying context as desired. The bounding box may bemodified in shape depending on the type of annotation. Here, boundingbox 401 contains some of ink 302 beyond the ends of highlight 305 butnot all of ink 302. Likewise, underline 306 and a portion of ink 303 isdesignated by bounding box 402. Bounding box 403 is different in that itincludes all of ink 304. Here, annotation 307 is a pair of margin bars.The system interprets the margin bars as referencing the horizontalstretch of ink 304. Of course, the horizontal selection in bounding box403 may include more than one line as determined by whether bars 403extend for more than one line. Bounding box 405 encompasses image 308and loop 309. Bounding box 404 encompasses loop 311 and connected note312.

In one aspect of the invention, bounding boxes 401-405 may be renderedto a user to show the user his annotations, thereby providing the userwith a quick understanding of the relevant portions (according to him)of the document. All bounding boxes may be rendered separately.Alternatively, bounding boxes may be combined based on various criteria.For instance, the bounding boxes may be combined based on shortdistances between the bounding boxes. FIG. 4 shows distancedeterminations being made between each bounding box and its neighboringbounding boxes.

FIG. 5 shows grouping of annotations in accordance with aspects of thepresent invention. Based on the distance determinations from FIG. 4, thesystem determines that the distance between bounding boxes 401 and 402is within a threshold and groups bounding boxes 401 and 402 into alarger bounding box 501. Combining bounding boxes may be done based ondistance determination, type of annotation, type of content having beenannotated, user identification, tablet identification, penidentification used to make the annotation, and the like.

FIG. 5 shows the resulting annotations being displayed to a user withunderlying contextual information in the bounding boxes. The boundingboxes may be ordered as appearing in the document or may be orderedbased on other criteria. The bounding boxes may be presented as boxes,with or without borders. To further distinguish the bounding boxes withannotations and related context information, additional visualembellishments may be used including shadows and/or tom edges.

Aspects of the present invention may be implemented as a shell in anoperating system. Alternatively, aspects of the present invention may beimplemented as part of an application running on the operating system.When applied at an operating system level, aspects of the presentinvention permit clippings to be gathered across documents fromdifferent applications. For instance, one may highlight a first documentin a web browsing application, underline a second document in a wordprocessing application, circle data in a spreadsheet application, andadd text and arrows to a mapping application. Next, these variousannotations may be shown together in a single view. Of course, moreviews may be permitted based on filtering. However, this examplehighlights at least one advantage of deploying aspects of the inventionon a system-wide basis.

An illustrative process relating to an operating system-level clippingview may include creating an annotation and storing the data from theannotation in a location accessible by the operating system. Next, auser may start a user interface that allows the querying and filteringof the stored annotations. When the user requests a view of theannotations and context, the operating system builds a list of theannotations and context (referred to as clips) from the annotation datastored above.

FIG. 6 shows annotations on electronic ink being grouped in accordancewith aspects of the present invention. Annotations may take a variety offorms. However, when the type annotation takes the same form as thecontext it is annotating, determination of what is context and what isan annotation can be difficult. In accordance with aspects of thepresent invention, what is considered to be an annotation may bedetermined based on the type of annotation, the shape of the annotation,the underlying context and the like. FIG. 6 shows a document inelectronic ink. Highlights 601, 602, 603 and 604 may be consideredannotations separate from the underlying ink document even though thedifference between the highlight and the original ink may be as small asa user using a different color and/or a different width pen tip. Otherdistinctions may also be used to determine the nature of an ink stroke(content or annotation).

FIG. 6 shows bounding boxes 605 and 606 combined into bounding box 609and bounding boxes 607 and 608 combined into bounding box 610. Theresulting boxes 609 and 610 with context and annotations may bedisplayed.

The user may initiate the display of the annotations and contextinformation by pressing a button or making another selection.Alternatively, an application may display the annotations and contextinformation to the user when the user starts the application. Bothapproaches may be used together or separately.

Processes for Handling Annotations and Clips

FIGS. 7 and 8 show processes for combining and displaying annotations inaccordance with aspects of the present invention. FIG. 7 shows twosteps: combine 703 and display 706. In step 703, annotations, context,and/or bounding boxes are reviewed to determine whether they may becombined. If so, they are combined, if not, they are left separate. Instep 706, the annotations, context, and/or bounding boxes are displayedto the user. For purposes of description, the information shown to theuser is referred to as clips.

FIG. 7 shows optional steps. The information that creates theannotations and/or context may be created by a user in step 701 anddirectly forwarded to step 703. Alternatively, annotations and/orcontext may have been previously created and stored. Step 703 mayreceive the information from storage 702.

After combining annotations and context, clips may be displayed asdescribed above with reference to step 706. Alternatively, the clips maybe stored in storage 704. The clips may then be filtered in accordancewith filtering criteria (for example, the user may only want to seeclips that contain red ink, highlights, certain words, certain shapes,and the like). The system may include the ability to perform handwritingrecognition on annotations to determine if the annotations contain textand what the text is. This text may be searched in filtering step 705.Likewise, the system may be able to recognize various shapes and filteron these shapes in step 705.

In yet another aspect of the invention, storing in storage 704 may notbe performed and only filtering step 705 be performed after combiningstep 703 and display 706. Whether or not storage 704 is used, combiningbefore filtering provides the benefit of reducing the number ofindependent clips that need to be searched during filtering step 705.

It is appreciated that both steps 701 and 702 may be used when, forexample, previously stored content from step 702 is used with newannotations from a user from step 701.

FIG. 8 shows an alternative process to that of FIG. 7. Here, the systemfilters in step 803 prior to combining in step 805 and displaying theclips in step 806. One advantage of filtering before combining is thatit allows a user greater influence in what clips are combined and shown.For instance, if a user only wants to have clips of one highlightingcolor shown, then filtering before combining eliminates non-selectedhighlights from the clips to be shown to the user.

Optional steps are also shown in FIG. 8. First, the input to filteringstep 803 may come from a user having created the annotations and/orcontext in step 801 or may come from a storage where previously storedannotations and/or content resides in step 802. It is appreciated thatboth steps 801 and 802 may be used when, for example, previously storedcontent from step 802 is used with new annotations from a user from step801.

FIGS. 9 and 10 show orderings of clips of annotations in accordance withaspects of the present invention. In FIG. 9, clips are displayed inregion 901. The clips 1-N are displayed in a raster pattern. Any numberof columns may be used. A single column provides the ability to onlyneed to look in a single direction to find clips. Multiple columns canbe more efficient in providing the clips to a user who is using them toobtain an overall idea of the relevance of the document. In FIG. 10, theclips are arranged in two columns. Here, one may need to scroll to thebottom of the displayed region 1001 to read all of clips I-N then needto scroll back up to the beginning to start reading the next column ofclips (clips N+1 and N+2).

Clips and Links

Clips may be used to provide information regarding the annotations in adocument. Alternatively, clips may also be used to link the user back tothe region of the document from which the clip was created. A set ofdisplayed clips may come from one or more documents from one or moreapplications. Links associated with the clips may refer back to thosedocuments across the applications.

FIG. 11 shows various levels of links in accordance with aspects of thepresent invention. Displayed region 1101 includes clips 1, 2, N, andN+1. These clips may not link to any documents, may link to onedocument, or may link to multiple documents. Here, clip 1 1102 links toa relevant portion of document 1106 from which clip 1 1102 originates.Similarly, clip 2 1104 may link to a relevant portion of document 1106from which clip 2 1104 originates. Alternatively, as shown by brokenarrows, the clips 1 1102 and 1104 may link to the document 1106 ingeneral.

To show different documents being linked by clips shown in region 1101,clip N 1103 may link to document 1105.

FIG. 11 also shows displayed regions 1107 and 1109. In both of theseregions, clips 1, 2, N, and N+1 are shown. Here, however, annotationsare made on the existing clips as annotation 1108 in region 1107 andannotation 1110 in region 1109. Multiple sets of links mayor may notexist. First, no links may be permitted based on annotated clips. So,for instance, activating clip 1 in region 1107 with annotation 1108 maynot link to any location. Second, a first level of linking may bepermitted. As shown by the solid arrow from clip 1 with annotation 1110in region 1109, the clip 1 may link generally to the previous set ofclips in region 1101. Alternatively, as shown by the broken arrow fromclip 1 of region 1109, the clip 1 may link to the portion of region 1101from which clip 1 originates. Third, the system may support unlimitedlevels of linking. For instance, the link from clip 1 in region 1109 maypoint to document 1106, the origin of the context and annotation forclip 1.

FIG. 12 shows an illustration of active content being annotated inaccordance with aspects of the present invention. Web pages and otherdocuments may include content that changes over time. In some instances,the pages themselves change. In other instances, they include contentthat may be only temporarily visible to the user, yet nonethelessrelevant and annotatable. FIG. 12 includes a web page 1201 with activecontent or data 1202. The page 1201 includes annotation 1203 thatannotates the active content or data 1202. At least two options arepossible to maintain the context of the annotation. In a first example,the system may capture a static image of the content 1206 (as a bitmap,JPEG, GIF, PDF, EPS, or other format) and may keep the image of thecontent 1206 available for display as a clip including annotation 1207.In a second example, the active content 1202 may be captured and storedlocally to allow the content 1204 to be associated with an annotation1205. This may include downloading the current active content ormaintaining a link to the content that was displayed at the time ofannotation.

User Interfaces

FIG. 13 shows a user interface in accordance with aspects of the presentinvention. A user interface 1301 is provided that allows variousselections of the content. The ability to show clips, filter clips andwhat to filter clips on is shown in region 1302. The filtering criteriamay include highlights flags, annotations, underlining, ink color,location, stars, ink notes, bullets, arrows, loops, and selectable text.Region 1303 shows a user interface that permits one to select the sortorder of the annotations.

Data Structures

FIG. 14 shows information stored with relation to an annotation inaccordance with aspects of the present invention. A clip 1402 may beshown in region 1401. The clip 1402 may be stored in a variety offormats including HTML, XML, and other formats. The content of each clipmay be represented by the following items 1403:

-   -   Annotation data (may include electronic ink);    -   Type of document;    -   Color;    -   Shape;    -   Type of Annotation;    -   Author;    -   Pen ID;    -   Tablet ID;    -   Date created;    -   Date modified; and    -   Link or storage of context.

Aspects of the present invention have been described in terms ofillustrative embodiments thereof. Numerous other embodiments,modifications and variations within the scope and spirit of the appendedclaims will occur to persons of ordinary skill in the art from a reviewof this disclosure.

The invention claimed is:
 1. A method of displaying clips comprising thesteps of: displaying a document, the document stored at a sourcelocation; receiving, within a current access session, a first user inputthat creates a first annotation of displayed active first content fromwithin the displayed document, wherein there exists a link to the activefirst content, the active first content is non-static content, and thefirst annotation is displayed within the displayed document; receiving,within the current access session, a second user input that creates asecond annotation of displayed second content within the displayeddocument, wherein there exists a link to the second content, and thesecond annotation is displayed within the displayed document; storingthe first annotation as displayed in the current access session togetherwith a static image of the displayed active first content from thecurrent access session; storing the link to the active first content,wherein the link to the active first content refers to the sourcelocation; storing the second annotation as displayed in the currentaccess session together with an image of the displayed second content;storing the link to the second content, wherein the link to the secondcontent refers to the source location; responsive to a user request,combining the stored first annotation and the stored second annotationbased on a criterion; rendering an image by retrieving a clip comprisingthe combination of the stored first annotation and the static image ofthe active first content and the stored second annotation and the imageof the second content; and displaying the rendered clip, whereinselection of said first annotation in said clip accesses the activefirst content via the stored link to the active first content.
 2. Themethod according to claim 1, wherein said clip includes additionalcontent.
 3. The method according to claim 2, wherein said additionalcontent includes an additional annotation.
 4. The method according toclaim 3, wherein said clip is a combination of two or more boundingboxes, said bounding boxes surrounding said first annotation and saidadditional annotation.
 5. The method according to claim 3, wherein saidclip is a combination of two or more regions surrounding said firstannotation and said additional annotation.
 6. The method according toclaim 1, further comprising utilizing a storage for storing said firstannotation and an image of said active first content.
 7. The methodaccording to claim 1, further comprising utilizing a storage for storingsaid first annotation and said link to said active first content.
 8. Themethod according to claim 1, further comprising utilizing a storage forstoring said first annotation and said active first content.
 9. Themethod according to claim 1, further comprising: receiving at least twosets of an annotation and associated content; combining said at leasttwo sets; and displaying a combination of said at least two sets. 10.The method according to claim 9, further comprising filtering saidcombination of said at least two sets.
 11. The method according to claim9, further comprising storing said combination of said at least twosets.
 12. One or more hardware memory storage devices having a programstored thereon, said program for storing and accessing clips andcomprising the steps of: displaying a document, the document stored at asource location; receiving, within a current access session, a firstuser input that creates a first annotation of displayed active firstcontent from within the displayed document, wherein there exists a linkto the active first content, the active first content is non-staticcontent, and the first annotation is displayed within the displayeddocument; receiving, within the current access session, a second userinput that creates a second annotation of displayed second contentwithin the displayed document, wherein there exists a link to the secondcontent, and the second annotation is displayed within the displayeddocument; storing the first annotation as displayed in the currentaccess session together with a static image of the displayed activefirst content from the current access session; storing the link to theactive first content, wherein the link to the active first contentrefers to the source location; storing the second annotation asdisplayed in the current access session together with an image of thedisplayed second content; storing the link to the second content,wherein the link to the second content refers to the source location;responsive to a user request, combining the stored first annotation andthe stored second annotation based on a criterion; rendering an image byretrieving a clip comprising the combination of the stored firstannotation and the static image of the active first content and thestored second annotation and the image of the second content; anddisplaying the rendered clip, wherein selection of said first annotationin said clip accesses the active first content via the stored link tothe active first content.
 13. The one or more hardware memory storagedevices according to claim 12, further comprising: receiving at leasttwo sets of an annotation and associated content; combining said atleast two sets; and displaying a combination of said at least two sets.14. The one or more hardware memory storage devices according to claim13, further comprising filtering said combination of said at least twosets.
 15. The one or more hardware memory storage devices according toclaim 13, further comprising storing said combination of said at leasttwo sets.
 16. A system for displaying clips of content and annotationscomprising: an input device for receiving, within a current accesssession, a first annotation of active first content that is displayedwithin a displayed document, the document stored at a source location,wherein there exists a link to the active first content, the activefirst content is non-static, and the first annotation is displayedwithin the displayed document, wherein the first annotation is createdin response to a first user input that annotates the active firstcontent; the input device for receiving, within the current accesssession, a second annotation of second content that is displayed withinthe displayed document, wherein there exists a link to the secondcontent, and the second annotation is displayed within the displayeddocument, wherein the second annotation is created in response to asecond user input that annotates the second content; a processor forstoring the first annotation together with a static image of thedisplayed active first content from the current access session; theprocessor for storing the link to the active first content, wherein thelink to the active first content refers to the source location; theprocessor for storing the second annotation as displayed in the currentaccess session together with an image of the displayed second content;the processor for storing the link to the second content, wherein thelink to the second content refers to the source location; the processorfor, responsive to a user request, combining the stored first annotationand the stored second annotation based on a criterion; the processor forrendering an image by retrieving a clip comprising the combination ofthe stored first annotation and static image of the active first contentand the stored second annotation and the image of the second content;and a display displaying the rendered chip, wherein selection of saidfirst annotation in said clip accesses the active first content via thestored link to the active first content.
 17. The system according toclaim 16, further comprising a storage storing said first annotation andan image of said active first content.
 18. The system according to claim16, further comprising a storage storing said first annotation and saidlink to said active first content.
 19. The system according to claim 16,further comprising a storage storing said first annotation and saidactive first content.
 20. The system according to claim 16, wherein saidclip includes an additional annotation, and wherein said clip is acombination of two or more bounding boxes, said bounding boxessurrounding said first annotation and said additional annotation.